1. Field of the Invention
The present invention relates generally to the processing of video images and, more particularly, to techniques for deinterlacing video images.
2. Description of the Related Art
All major television standards use a raster scanning technique known as xe2x80x9cinterlacingxe2x80x9d or xe2x80x9cinterlace scanning.xe2x80x9d Interlace scanning draws horizontal scan lines from the top of the screen to the bottom of the screen in two passes. Each pass is known as a field. In the National Television System Committee (NTSC) standard used in North America, each field takes approximately {fraction (1/60)}th of a second to draw.
Interlace scanning depends on the ability of the cathode ray tube (CRT) phosphors to retain an image for a few milliseconds, in effect acting like a xe2x80x9cmemoryxe2x80x9d to retain the previous field while the newer interleaved field is being scanned. Interlace scanning provides a benefit in television systems by doubling the vertical resolution of the system without increasing broadcast bandwidth.
FIG. 1 shows a number of parallel horizontal scan lines 10 on a conventional television display. A first set of horizontal lines 12 is scanned in a first field period and then a second set of horizontal lines 14 is scanned in a second field period. Thus, the first field is temporarily shifted by {fraction (1/60)}th of a second from the second field. When rapidly changing images are being displayed, an object in motion may appear to be fuzzy due to the temporal displacement between the two fields.
This temporal displacement typically does not create a problem on conventional television displays, primarily because the image of the xe2x80x9colderxe2x80x9d field quickly fades in intensity as the light output of the phosphors decays. A secondary reason is that the spatial displacement in the images caused by motion results in a fine detail that television displays do not resolve well. For these reasons, interlace scanning of motion pictures works acceptably well on conventional television displays.
FIG. 2 shows a set of progressively scanned horizontal lines 16. In progressive scanning, all horizontal lines 16, are scanned out in one vertical pass 18, so there is no time displacement of adjacent lines as in interlace scan. Progressive scanning requires a much higher bandwidth signal. Consequently, progressive scanning is typically used for applications where improved image quality and higher resolution are required, relative to conventional television systems. Progressive scanning is widely used in computer CRTs and liquid crystal displays (LCD).
If a motion picture formatted for an interlaced monitor device as in FIG. 1 is to be displayed on a progressively scanned device as in FIG. 2, then it must be converted from the interlaced format to the progressive format. This format conversion is known as deinterlacing. FIG. 3 is an illustration of a deinterlace process 19 of the prior art. A first series of interlaced video fields 20 is generated by a video source (not illustrated) at {fraction (1/60)}th second intervals.
In this example, each of the video fields 20 has a spatial resolution of 720 horizontal by 240 vertical pixels. Each field contains half the vertical resolution of a complete video image. The first series of video fields 20 are input to a deinterlace processor 22, which converts the 720 by 240 interlaced format to a second series of video fields 24. In this example, each of the second series of video fields 24 may have 720 by 480 pixels where the fields are displayed at 60 frames per second.
FIG. 4 is an illustration of a prior art method of deinterlace processing, which uses field combination to deinterlace an image using multiple fields. A video field 26 containing scan lines 30, and a previous video field 28 containing scan lines 32 is fed into a field combination deinterlace processor 34. The result is a combined frame 36 with scan lines 38 sourced from video field 26 and scan lines 40 sourced from video field 28. When this simple deinterlacing of the prior art is performed, and a motion picture formatted for an interlace display is converted to a progressive format, a noticeable xe2x80x9cartifactxe2x80x9d or error arises because the image content of vertically adjacent lines is time shifted by {fraction (1/60)}th second as noted previously. The error is most visible around the edges of objects that are in motion.
FIG. 5 is an illustration of a prior art method of deinterlace processing, which uses line interpolation to deinterlace an image using a single reference field. The method 42 interpolates or doubles the number of lines of one field to produce a progressive frame. A video field 44 is scanned from an image to contain a half set of lines 46. The half set of lines 46 is deinterlaced by line interpolation in a deinterlacing interpolator 48.
The resulting frame 50 will have all the lines 46 of the original video field 44. The remaining lines 52 are created by interpolation of lines 46. The resultant image will not have motion artifacts because all the lines in the image will be created from lines 46 that are time correlated. This alternative method 42 of deinterlacing does not produce motion artifacts, but the vertical resolution of the image is reduced by half.
FIG. 6 shows a deinterlaced image 54 with a stationary object 55 that is rendered without distortion. FIG. 7 shows an image 56 with the object 55xe2x80x2 in motion. The edges of object 55xe2x80x2 create artifacts 57 on the edges of the image 54 because of the aforementioned temporal shift. These artifacts 45 are introduced into the image by the conventional field combination deinterlacing method 25 of FIG. 4.
Motion artifacts in deinterlaced video images will have characteristics that vary depending on the original source of the motion picture. Video cameras, such as those used for television broadcasts, update motion in each field produced so that fields at 60 Hertz (Hz) will have motion updated at 60 Hz. But motion pictures from other sources are commonly displayed as video, which requires a conversion a different type of motion picture to video. For example, movies originally shot as film must be converted to video for display on a television set. Since film is originally shot at 24 frames per second, maximum update rate for motion for a film source is 24 Hz.
A film source may be viewed as a series of still image frames that are displayed in series at the rate of 24 per second. Film is converted to video in a two step process. First, each of the original film frames must be converted to video fields. Secondly, the video fields must be sequenced in a way that allows them to be displayed at 60 Hz for the NTSC video standard or at 50 Hz for the phase alternation line (PAL) standard. The conversion from film to NTSC video is known as 3:2 pulldown. The conversion of film to PAL video is known as 2:2 pulldown.
FIG. 8 illustrates a diagram of a method 58 for converting film into video of the prior art. A film frame 60 is digitized according to a standard for digital video known at ITR-R BT.601, which specifies the number of horizontal samples per line and the number of lines per field. The film frame 60 is shown with a horizontal by vertical resolution of 720xc3x97480, which is the approximate resolution specified by ITU-R BT.601 for NTSC video. (Note: this discussion assumes that film is being converted to a digital format. This may not always be the case but the explanation of 3:2 or 2:2 pulldown applies whether or not the film is digitized.)
Each film frame 60 contains the fall vertical resolution of the image. Since video fields contain half the vertical resolution, each film frame will be converted into two video fields. Assuming the horizontal lines of frame 60 are numbered in sequence from top to bottom, the white bands indicate even numbered lines 62 in the original film frame and the gray bands indicate odd numbered lines 64. When the frame is converted to fields, the even lines 62 are assembled into a first video field 66 and the odd lines 64 are assembled into a second video field 68. The field 66 is shown as white and the field 68 is shown in gray in the figures to indicate whether they contain the even numbered lines 62 or odd numbered lines 64 of the original frame 60.
The significance of the method 58 is that the video fields 66 and 68 created from the original film frame 60 are time correlated. In other words, the image contained in the film frame 60 is a xe2x80x9csnapshotxe2x80x9d in time; so there is no motion displacement between the even field 66 and odd field 68 that are derived from the same original film frame 60. This is not the case with video that was originally shot with video cameras, such as those used for television. Video cameras update motion in every field, and there are no xe2x80x9cframes.xe2x80x9d As mentioned previously, film has 24 frames per second and NTSC video uses 60 fields per second. The ratio of 24:60 equals 2:5. This means that for every 2 frames in the film source, 5 video fields must be created.
FIG. 9 illustrates a prior art method 70 providing for 3:2 pulldown (converting film frames to NTSC video fields). A time sequential series of film frames 72 at 24 frames per second are converted to even and odd fields. The fields are then displayed as video fields 74 at 60 fields per second. From FIG. 9 it is illustrated that Fields 1, 2, and 3 originate from Frame 1; Fields 4 and 5 originate from Frame 2, and so forth. Therefore, five fields are created from two film frames in order to achieve the 24/60 ratio of film frames to video fields.
The fields 74 are then assembled into a series of deinterlaced frames 76 by the method 46 shown in FIG. 5, which performs deinterlacing by combining the most recent two video fields into a single frame. Fields 1 and 2 are combined to create DI Frame 1; Fields 2 and 3 are combined to create DI Frame 2; Fields 3 and 4 are combined to create DI Frame 3, and so forth. Since Fields 1 and 2 came from the same original film frame, there will be no motion artifacts in DI Frame 1 because Fields 1 and 2 are time correlated. There will be no motion artifacts in DI Frame 2 because Fields 2 and 3 come from the same film frame and are time correlated.
However, DI Frame 3 is the combination of Fields 3 and 4. Because Field 3 comes from film Frame 1 and Field 4 comes from film Frame 2, there may be motion artifacts because the two fields come from different original film frames. The diagram in FIG. 9 shows a heavy border around the deinterlaced frames 76 that result from the combination of fields from different original film frames and may have motion artifacts. The deinterlaced frames 76 with light borders will not have artifacts.
FIG. 10 illustrates a prior art method 78 providing for 2:2 pulldown (converting computer rendered animation to NTSC video fields). The animation motion picture forms a series of frames 80 at a rate of 30 frames per second. The series of frames 80 is converted to video fields 82 at a rate of 60 per second. 2:2 pulldown is also used to convert film at the rate of 24 frames per second to PAL video fields at 48 fields per second. The PAL video fields are then displayed 50 fields per second.
FIG. 10 further illustrates that when the video fields 82 are converted by a deinterlace processor by combining the two most recent field pairs into series of frames 84, that motion artifacts may appear in half the deinterlaced frames. DI Frames 1, 3, 5, 7, and 9 are created from pairs of fields that originally come from the same frame and so will be free of artifacts, but DI Frames 2, 4, 6, and 8 are created from fields that come from different original frames and can have artifacts.
The 24 frame per second series of frames 72 of FIG. 9 and the 30 frame per second series of frames 80 of FIG. 10 have a characteristic in common: they are types of motion pictures sources that have full resolution frames from which time correlated even and odd fields are derived during the conversion to video. This type of source is referred to in this document as a xe2x80x9cprogressive frame source.xe2x80x9d
The purpose of a deinterlace processor is to create progressively scanned frames from a series of interlaced video fields. If the original source of the motion picture is a type that has time correlated fall resolution frames, then knowledge of this type of source will be useful to a deinterlace processor. In view for the foregoing, it is desirable for a deinterlace processor to have a method for determining the original motion picture type from the series of input video fields, so that the deinterlace processing may be optimized for the source type.
The present invention fills these needs by providing a method and apparatus for deinterlacing a video input stream by analyzing the video stream to determine if the video stream originated from a progressive scanned source. It should be appreciated that the present invention can be implemented in numerous ways, including as a process, an apparatus, a system, a device or a method. Several inventive embodiments of the present invention are described below.
In one embodiment of the present invention, a digital image processor is provided. The digital image processor includes a deinterlacing processor that is implemented upon a digital processing unit. The deinterlacing processor is coupled to an input operable to receive an interlaced video stream, a digital memory for storing portions of the interlaced video signal, and an output operable to transmit a deinterlaced video stream. The deinterlacing processor is operable to detect the source type of the received interlaced video stream to generate the deinterlaced video stream having reduced or no motion artifacts.
In another embodiment of the present invention, a method for deinterlacing an interlaced video stream is provided. The method includes receiving a set of video fields from an input of the interlaced video stream. The set of video fields is analyzed to determine a source type of said interlaced video stream. If the source type of the interlaced video stream contains progressive frames, then the sequencing the interlaced video stream is detected. The set of video fields is then assembled into a deinterlaced video frame with reduced or no motion artifacts using the source type and the sequencing of said interlaced video stream as a control.
An advantage of the present invention is that it allows for reduction of motion artifacts in video images. By identifying the type of the source motion picture, a video deinterlacer can use the information to combine frames that are time correlated. The result is a video image that is free of artifacts.
Other aspects and advantages of the invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the invention.